Brainwave: A Multidisciplinary Journal (ISSN: 2582-659X), Vol. 3, Special Issue, March 2022, pp. 48-58, © 
Brainware University 


Face Recognition based Attendance System 
Swarup Chakraborti*, Sanjukta Mishra* 


Department of Computer Science and Engineering 
Brainware University 
*Corresponding Author 


Abstract 


The attendance process is essential in any of the organization like School, College or any organization. Manual attendance 
process is very tedious and time consuming process. Face is one of the most individual identities that distinguish each 
other. Face feature of each person is different than other. This system will automatically track the attendance system of 
an employee. Proposed system will change the manual system and convert it to digitized system. Face recognition is one 
of the most popular and efficient application in image processing and also used in digital camera and mobile phone also. 
In this proposed system YOLO V3 algorithm is used for face detection. The main part of this system is a camera installed 
in front of a entry point of a door. Through this camera, face of the employee is captured first and then compares with 
previously stored face database and check the feature and identity of the face. When the face is identified employee's 
attendance will be taken automatically and necessary information will be recorded in the CSV file. If an employee is newer 
in an organization, his / her ID, Name will be inserted into the system first and then his face image is collected from 
camera and stored in the face database. Face feature is extracted through binary classifier of Convolution Neural 
Network. This binary classifier will check if he/she is wearing a eye glass or not or he/she has any spot in face or not or 
his / her face has any deformities or not. 
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Introduction 
All organization maintains an attendance [1] system of his employees. It is an essential task to find 
out the employee’s individual performance, regularity for the entire month as well as year. 


Draw Back of Manual Attendance System 
In a manual attendance system there are many problem arises. 


@ It is time consuming process 

@ More man power is required 

@ Attendance book may be stolen 

@ There is a chance of forge and erroneous 


Scope of this System 


This smart attendance system can recognize the identity of each individual and record the face in a 
face database. The scope of the project is 


@ Staff of an organization are the target group of this system. The facial recognition process can 
only be done for one person at a time. 

@ Staff ID, Name are stored in a CSV file and attendance is also recorded in CSV file in daily 
basic with filename of current date. Automatic face recognition is also used in various 
devices like mobile phone etc. This face detection technique can also be used to find out the 
criminal from the CCTV footage and compare with criminal database. 

@ Face recognition is a task of identify a face which predefined by the system. The entire 
processing concepts involves under the Digital Image Processing and Computer Vision 
concept. 

Introduction of Face Recognition and Attendance System 


Face Recognition process has two parts. 1) Face Detection and Feature extraction 2) Face 
Recognition from Dataset. For detecting a face Viola [5] V3 algorithm is used to detect the face 
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from the live stream or images. Feature of the face is extracted with the help of Local Binary Pattern 
Histograms (LBPH) and stored. Face Reorganization processing also identifies the Spectacle on an 
eye and also identify if the face format is normal or abnormal. To find the Spectacle on an eye I 
have used Convolution Neural Network (CNN). The Convolution Neural Network uses Binary 
classifier returns 0 or 1 value according to the feature of Training and Testing Dataset. Face 
formation is also identified by Convolution Neural Network (CNN). Name and ID of an employee 
is entered in the system at the first time when a face is recognized by the system. Above statement 
is shown in the Figure 1.1 . 


Face Recognize and Feature Extract 


Figure 1 .1: Operating process of face reorganization 
Attendance of an employee of an organization is automatically recorded in a CSV file, if his/her 
face is already recognized by the system, his/her Name and ID will be shown with his/her face 
image. Every day a new CSV file will be generated by the system and current date will be the file 
name of the CSV file. CSV file will contain the Name of the Employee and his /her ID and also 
insert the status Present / Absent. This proposed system is less costly that other systems and also 
efficient. 


Digital Image Processing Concept 


An array of sensors are used to acquire the image in a digital form. So Digital image [10] can be 
represented as a two dimensional space through a function fx(x,y), where x and y are the coordinate 
of the image. In a digital image processing image is also represented as a matrix form which is given 
in the Figure 1.2. So a gray scale image can be represented in 2D space. 


f<(0.0) £x(O.1) £x(0.2) £x(O.N-1) 
f<(1.0) fx(1.1) fC.2) fC.N-1) 

fx(xy)= 
> fx(2.0) fx(2.1) fx(2.2) fx(2.N-1) 


f<(3.0) fx(3.1) £&G.2) f&G.N-1l 


fx<(M_1.0) fx(A1-1.1) fx(1-1.2) f(1-1.N-1) 


Figure 1.2 : Matrix format of an image 


Relation between Pixel and Color 
An image contains a number of picture elements, called Pixel. In a black and white or gray 
scale image, each pixel contains only one value, either 0 (absent of light) or 1 (present of 
light). In gray scale image, intensity range of each color is between 0 to 255. So the 
dimension of an image is = number of rows X number columns x color channel. For 
grayscale and black and white images, color channel is 0. In RGB images color channels 
are 3. Image resolution is also uses in bit per pixel value. The general Color image format 
is RGB. RGB has Red, Green, Blue color channels. Intensity range of each color is from 0 
to 255. So in color image each pixel have 3 color channels and each color channel is 
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represented with 8 bit per channel. So total bit depth of each pixel is 3x8=24 bpp. 


Literature Review 


Face Recognition Technology is one of the most popular Technology in the subject of Image 
Processing and Computer Vision. It identifies Human Face from digital Image or from Video Frame 
and matches it with predefine database of faces. It identifies the face by extracting the previous face 
feature of same image and compares it with current face of the current image. It is basically used to 
authenticate the user through ID verification service. Today this technology is used in various 
applications. Face Recognition Base Attendance system is another important technology which 
identifies the face of a person of that organization and marks the attendance digitally. Face of that 
person is already stored in the system. 


Kanade proposed the facial recognition technology[12] in 1977. In America, The Defense Advanced 
Research Project Agency (DARPA) and the Army Research Laboratory (ARL) established the face 
recognition technology in 1993, After that various research work was started on this technology. 
One of the most popular and efficient technology was discovered in 2001 by Paul Viola[3] and 
Michael Jones, a Real-time face detection technique from video footage. It basically uses the 
AdaBoost Technology. 


Yohie Kawaguchi et.al [3] was designed a system that was used to continuous observation and using 
face recognition. They were developed an Realtime student detecting method. In this system two 
cameras were used. One camera was used for sensing camera, that was used to identify number of 
seats inside class and another camera was used to capture the face. The two cameras were installed 
in the classroom wall. 


In this paper author [4] used the Viola Jones [5] algorithm for face detection. Here author have 
considered several parameters of face image like sharpness, brightness and resolution of image and 
also considered the head position, Convolution Neural Network is to map the dimensionality of the 
image. 

N. Kar [11] has used a new approach in a Face Reorganization based automated attendance 
management system .Author used the Principal Component Analysis (PCA) and FLTK(Light Tool 
Kit) and OpenCV [9] library. Haar Cascade is used access the frontal image. In this algorithm after 
cropping the face image, Principal Component Analysis is applied to recognize the face with the 
measuring the distance. 

Onur Sanli and Bahar Ilgen [7] published a paper in Springer Journal on the topic of ‘Face 
Recognition based attendance System’. Sikandar Khan , Adeel Akram , Nighat Usman [6] (Springer 
Nature), another paper was published on ‘Real Time Automatic Attendance System for Face 
Recognition’. This system used Face API and OpenCV [9]. Both of the papers have proposed a 
system for automatic attendance management of class students of an educational institute on the 
basic of facial recognition. Authors of this paper [7] uses Principal Component Analysis (PCA), 
LBA based system to recognize the face and used Haar cascade classifier to identify the faces from 
the classroom camera. 


These system will access multiple faces at a time through CC Camera, which is installed in the class 
room and the find out the every single face from the group faces and the recognizes the faces with 
predefine face database and at last attendance system is recorded. For face identification purpose 
both of the system used Haar Cascade Classifier ( Viola-Jones Algorithm) for identifying the face. 
Here face is captured first and then cropped the face. After cropping the face some background is 
removed. The cropped face is converted to grayscale image and stored in the face database. Here 
two files were used to store the data. One file stores the students’ information of a class and another 
file is used to keep track of the attendance. 


In this system the attendance will be recorded only for employees of an organization. Here only 
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single face will be recognized at a time, instead of multiple faces. This digital attendance system is 
very easier and less time required and also effective. If an employee is new in an organization, 
his/her attendance will not be recorded until his/her recognized face image is inserted into the 
system. First time when an employee’s face is recognized, system will capture the various facial 
features from the image and stored it the Userdetails.csv file with user ID, Name and facial 
characteristic. At the time of attendance, image of the employee’s face is captured and compare with 
stored image in the image database. If the face feature is matched with system his attendance is 
granted and stored in separated CSV file. Every day a new attendance file will be generated with the 
filename of current date. 


Algorithms for image processing 


In this proposed system we have used Viola-Jones Algorithm i.e CascadeClassifer, to detect the 
face. Local Binary Pattern Histogram (LBPH) is used to extract the feature of face for recognizing 
the face. Convolution Neural Network (CNN) is used to eye glass detection and also face format 
detection. 


Algorithm of Viola-Jones 


Violo-Jones algorithm is a very popular algorithm for Face detection and Haar cascade classifier, 
which is available in OpenCV library, is used to detect the face.This algorithm takes 20 steps of 
comparison to find out if the image has a face or not. AdaBoost Technology is used to remove the 
redundant features and extract the actual feature. The matching process is done on grayscale image. 
So this algorithm has four stages. The stages are 

a) Haar Feature Selection 

b) An Integral Image Creation 

c) Use of AdaBoost Technology 

d) Cascading Classifier 
Below figures 3.1 shows the Edge, line and center surround features of Haar Cascade Classifier. 


lave DM=a¢ede 


Edge Feature Line Feature 


a ® 


Center surrounding Feature 
Figure 3.1: Haar cascade edge, line ,center-surround feature extractor 


The formula of the above technology is given below 

Q(X)= — Ti * gi(X) (1) 
Here Q(X) is the strong classifier and Ti is the weight of each weak classifier qi(X). Through this 
algorithm specific objects are find out from images or video frames after scanning the similar shape. 


Local Binary Pattern Histogram (LBH) 


LBH is one of the simple and efficient face recognition algorithm. Ojala et al. [15] was the inventor 
of this algorithm. It detect the frontal faces and side face also, Pixels of the neighbor of an image is 
labeled using texture measurement operations with threshold values. A tag is created for each pixel 
with value of 1 or more 0. Labels are used to compare the adjacent pixels to the center of pixel NxN. 
Here LBPA,R is defined by three different circular neighborhoods. Here ‘A’ represents the number 
of neighboring pixels and ‘R' represents the sampling Radius as it is shown Figure 3.2. 
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FIGURE 3.2: VARIOUS CIRCULAR LBP P,R OPERATOR 


In this algorithm[24 four parameters are used. 

Radius: Around the circle a local binary pattern is build up, bas on the center of a pixel, set the value 
1. 

Neighbors: It is the number of data points, that build up a circular local binary pattern. 


Grid X : It is the number of cell in the Horizontal direction. Generally the value is 8. 


Grid Y : It is the number of cell in the Vertical direction. Generally the value is 8. 
The formula of LBP(A,R) with n x n matrices is given below 


jot = l,y=>O 
WY? = 1 Oo, y<O 


——g A | 


(2) 


u(x, — x-)2* 
A 0) 

LBP(A.R2)= (3) 
Here y is the difference between the center pixel and its neighboring pixel. Here center pixel is xc 
and xA is the neighbor of center pixel. Here u(y) is the bit produced by LBP operator. 
Convolution Neural Network (CNN) 
Like Biological neuron artificial neuron and its network has build up depending upon the biases and 
along with learnable weight. Convolution Neural Network (CNN) is _ based on artificial neural 
network and entire concept is built on the basic of the algorithm of Deep Learning.. So the four 
important layers of Convolution Neural Network are 

e) Convolution Layer- defines filters, number of input channels, additional hyperparameters 

f) Activation function is used in ReLU (Rectified Linear Unit) layer 

g) Pooling Layer- reduce the dimensions of data 

h) Fully Connected Layer — used to connect every neuron of each layer 


Result 
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In this System Face of an employee of an organization will be inserted into the system first with 
his/her Name and ID. 


Enter Number1 

Enter ID23 

Enter NameAmal 

ID= <built-in function id> Name= Amal 


0 100 150 


FIGURE 5.1: FACE IMAGE OF A NORMAL PERSON FIGURE 5.2: AFTER IDENTIFYING THE FACE 


System will recognize the face and its feature like he/she is wearing a glass or not, his face is normal 
or some abnormality is shown in the face. His/her ID, Name and characteristic of face will be stored 
in a CSV file. Here the face is normal face no spectacles is shown. Here some of the Identified faces 
are shown below 

The Figure 5.1 of the document shows that an image is captured first with his/her ID and Name. Then 
from this image face is identified and shown in Figure 5.2 and stored in the image folder. After 
identifying the face various face features and face type is identified by the system and stored in the 
CSV file. Various Feature of the face is shown below. 


((1-]] 

No Glass 

MoSpot 

Normal Face 

['23', "Amal’, @, ‘NoSpot’, ‘Regular’ ] 


Figure 5.3: Various features of face 


This face image shows an abnormal face image. System will capture the face of a person after entering the 
ID, Name of that person. 


Enter ID68 
Enter NameKuntal 
ID= <built-in function id> Name= Kuntal 


Figure 5.4: Face image of an abnormal perso figure 5.5: After identifying the abnormal 


face 
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No Glass 
Glass value= [1-] 


[[e.]] 
NoSpot 
[[e.]] 


Irregular Face 
['68", "Kuntal’, ‘NoGlass', ‘NoSpot', ‘Irregular’ ] 
Figure 5.6: Various feature of identified face 


After identifying the face of a person, system finds out the different features of face, which is show 
below. 


Below Image shows that the person is wearing a Spectacle and his face has identified accurately 


Enter ID33 
Enter NameJoy 
ID= <built-in function id> Name= Joy 


Figure 5.7: Face image, wearing spectacle figure 5.8: After identifying the spectacle 


face 
After identifying the face system has find out the several features of face, which is given below. 
[[@.]] 
a Glass 
Glass 
Glass value= [1.] Glass value= [0.] 
[[1.]] [[@.]] 
NoSpot NoSpot ; 


Figure 5.9: Various feature of identified face wearing spectacle 


After identifying the faces and the face feature, the image of the face is stored in a folder and all the 
above information is stored in the Userdetails.csv file. Figure 5.11 shows the image folder and 
Figure 5.10, shows the content of Userdetails.csv file. 


Bg Facerkecattendance UserDetails.csv X< aul 
~ fh Trainingimage 1 to 5 of 5 entries [ Filter | 
. Id Name Glass FaceSpot Facetype 

b Alen.1 JPY 1 Alen NoGlass NoSpot Regular 

& Bimal.5.jpg a Sumit NoGlass NoSpot Regular 

3 Dilip NoGlass NoSpot Regular 

Ba Dilip.3.jpg 4 Amal NoGlass NoSpot Regular 

S it.2j Ls Bimal NoGlass NoSpot irregular 

% ae <- Show /10 ~]|per page 
Figure 5.10: Face image stored Figure 5.11: Information of a person stored in a file 
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The face is recognized of the above images and identified. Attendance process of the above persons 
are given below. Here 


2021-07-tlesv X ill 


Normal Face 
['3-Dilip'] 
Not Ok 


No entries “itr 
3 (‘Dilip’) Present 


Show {10 v]per page 


Figure 5.12 : Shows the recognized face with id and name and corresponding attendance in csv file 


Persons appear second time in the system and his face is already recognized by the system. 


['68-Kuntal"] 
Not Ok 


Ti [‘Kuntal’] Present 


Figure 5.13 : Shows the recognized face with id and name and corresponding attendance in csv 
file 


System will recognize the face and his / her ID and Name will be shown on his image and his/her 
attendance will be recorded. 


Analysis & Discussions 


System has identified the face and recognized the face on the basic of some predefined model. The 
model has trained properly and below figure shows the training process of the model. 


Enter your Choice 
Enter Number3 
/content/drive/MyDrive/FaceRecattendance/TrainingImage/.ipynb_checkpoints 
/content/drive/MyDrive/FaceRecattendance/TrainingImage/ Alen.1.jpg 
/content/drive/MyDrive/FaceRecattendance/TrainingImage/ Sumit.2.jpg 
/content/drive/MyDrive/FaceRecattendance/TrainingImage/ Dilip.3.jpg 
/content/drive/MyDrive/FaceRecattendance/TrainingImage/ Bimal.5.jpg 
Image Trained 

Figure 5.16 : Shows the training face of the model 


Here the training phase of the Convolution Neural Network (CNN) is shown below. The network used 
binary classifier with epoch value 20. The training process is shown with training and testing dataset. The 
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accuracy level is 75%. I have used 15 images for training and testing purpose. If one thousand or more 
images are used for training and testing purpose, accuracy level will increase. 


Figure 5.17: Training of a binary classifier of convolution neural network 


Two graphs are shows below. The top graph shows cross-entropy loss over epochs on the basic of 


train 
Cross Entropy Loss 


2 Classification Accuracy 


atl 


F IGURE 5.18: CONVOLUTION NEURAL NETWORK GRAPH AND ACCURACY LEVEL 


(blue) and test (orange) dataset. Classification accuracy over epochs is shown in the below graph. 
Here accuracy rate shows 80% accurate. 


Future Scope 


@ This system can be modified in such a way that it can track the visitor in an organization or in 
a school or college with the digital registration process. It can also be modified in such a way 
that it can be used home security purpose. 


@ This system can also be installed in the bank ATM. The ATM machine rooms contains CCTV 
camera. This camera will be equipped with face reorganization system and connected with 
central server of the bank and also connected with ATM machine. The customer will only 
access his/her account and draw money from ATM machine, if his face image is match with 
stored image with the bank database. It will help to prevent ATM hack, fraud and also increase 
the security of ATM. 

@ Any government organization can verify a person with recognizing his/her face with AADHAR 
data or organization can collect a person’s AADHAR information after recognizing the face. 

@ Updating of the face image is also required due to the change of facial characteristics. Facial 
characteristic can be changed due to some disease or due to some accidental purpose. At the 
time of face detection face alignment, orientation and occlusion must be considered and must 
be implemented in the above system. 

@ In future with face recognition system can also use iris or retina recognition using the thermal 
spectrum use in the applications gives a very high degree of accuracy. In this system Local 
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Binary Pattern Histogram (LBPH) has used for face recognition and training purpose. 


@ Thermal scan of the image is also required due to COVID 19 situation. System will 
automatically recognize the employee’s face as well as his/her body temperature at the time of 
attendance. If his/her temperature is not normal his/her attendance will not be granted. This 
type of system can also be implemented at the entry and exit point of the airport, where 
passenger is entering in the airport to ride the plane. 


Limitations 
All the system may not be hundred percent accurate and cannot be full proof system. Every system, 


it may be the commercial software or other system, there must have some limitation. In our system 
there are some limitations. Limitations of the system are given below: 


@ Low resolution image is not suitable for face detection purpose. It will access single image 
at a time. Small image size will not work properly in a face recognition system. 
In this system change of occlusion of face and orientation is not considered. 


This system runs on digital platform, so every time continuous power supply is necessary. 
To avoid the power cut problem a power back battery is necessary. 


e 

e 

@ Slow Computer and old Operating System and less memory and less storage location will 
not support this attendance system. The Operating System must be 64 bit version. 

@ If the database or computer is corrupted then entire system will be in vain. 

@ 


Camera will be placed in proper angel and position where face will be captured. 
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